In many instances, video transmission systems are prone to transmission errors in the form of the corruption or loss of one or more transmitted pictures or frames. The effects of transmission errors are compounded due to the use of motion-compensated temporal prediction or predictive coding. Predictive coding is the process of predicting the contents of some, or a majority, of the image frames or pictures in a video sequence based on the other, previous frames or pictures in the sequence. Predictive coding is advantageous because it provides high compression efficiency. However, because pictures or frames are often used to predict subsequent pictures or frames, transmission errors not only affect the decoding quality of the current picture, but are also propagated to the following coded pictures. This is referred to as “temporal error propagation.”
Several techniques have been developed for reducing or eliminating temporal error propagation. These techniques include both interactive and non-interactive methods. Interactive methods require that the recipient transmit information about the corrupted decoded areas and/or transport packets back to the transmitter (i.e., feedback). Where feedback is not possible, or preferable, non-interactive methods, such as forward error correction (FEC) or the use of redundant pictures, may be utilized.
The use of redundant pictures has been standardized by H.264/AVC or simply H.264 or AVC. H.264 was developed by the joint video team (JVT) of the ISO/IEC Motion Picture Experts Group (MPEG) and the ITU-T (International Telecommunication Union, Telecommunications Standardization Sector) Video Coding Experts Group (VCEG).
A redundant picture is a redundant coded representation of a picture, referred to as a primary picture, or a part of a picture (e.g., one or more macroblocks). The redundant picture may be considered the same temporal representation of the information content of the primary picture. Each primary coded picture may have as many as 127 redundant pictures associated therewith. If the region represented by the primary picture is lost or corrupted due to transmission errors, a correctly received redundant picture containing the same region may be used to reconstruct the region.
While the use of redundant pictures may be beneficial, it also increases the overhead associated with video encoding and transmission, as well as detrimentally affects the overall coding efficiency. It would, therefore, be beneficial to derive a method of allocating fewer or no redundant pictures to one or more primary pictures in order to cut down on the number of pictures that would need to be encoded and transmitted, but without reducing or eliminating the temporal error propagation-reducing benefits of using redundant pictures.
One such method has been proposed to the JVT by Miska M. Hannuksela and the named inventors of the present application (Zhu et al., Coding of Redundant Pictures for Improved Error Resilience, Joint Video Team (JVT) of ISO/IEC MPEG & ITU-T VCEG (ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6) 18th Meeting: Bangkok, Thailand, 14-20 Jan., 2006, hereinafter “JVT-R058”). According to the method of JVT-R058, a hierarchical method is used for allocating redundant pictures to only certain primary pictures. In particular, for each group of pictures (GOP), a redundant picture may be coded for the first picture in the GOP using the first picture of the previous GOP as a reference. In the case of the very first GOP, either an intra coded redundant picture or no redundant picture may be allocated for the first picture. According to JVT-R058, the GOP may further be divided into two or more sub-GOPs, and for the first picture of each latter sub-GOP (i.e., each sub-GOP following the first sub-GOP) a redundant picture may be coded using the first picture of the previous sub-GOP as a reference. As described above, the allocation method of JVT-R058 may be limited, however, because it is fixed, assuming that the number of pictures in a group of pictures (GOP), as well as the sub-GOP hierarchy, is fixed. In other words, the same primary picture(s) of a GOP are allocated one or more redundant pictures in all instances.
The method of JVT-R058, therefore, does not take into consideration the fact that certain pictures of the GOP, or of the overall video sequence, may be more sensitive to transmission error or corruption. In particular, there are often certain pictures where the loss or corruption of these pictures may cause more distortion to the overall video data than the loss or corruption of other pictures. Because of this, more protection should be afforded to these, more sensitive, pictures.
A need, therefore, exists for a mechanism for adaptively allocating redundant pictures, wherein pictures that are more sensitive to transmission loss or corruption are better protected.